National Institute for Learning Outcomes Assessment 

May 2016 


Higher Education Quality: 

Why Documenting Learning Matters 


A Policy Statement from the National Institute for Learning Outcomes Assessment 



nowledge accountability connection self-reflection edu 

igenuity intellect curiosity challenge create achieve? 

uality innovation success ingenuity intellect curil|^ 
ccess quality innovation success ingenuity self-refl^HL 
ducate action understand communicate curiosity clH 
Dnnection self-reflection knowledge accountabilitv HEV 
inovation success ingenuity intellect curiosity b 

nowledge accountability connection self-refle ^ 

Dlf-reflection understand communicate listen » /V C 
Dmmunicate listen learn access quality innoi 
uality self-reflection curiosity challenge 
nderstand intellect knowledge accounte 
Election educate action understand com 
nowledge accountability connection self-i 
hallenge create achievement conne 
ccess quality action create achieve / 

jccess educate action communicat A 

Election knowledge accountability < \ ' 

>arn access quality innovation succi^ 


nicate listen learn access quality innovation success 
■l^wtion understand communicate listen learn access 
^J^connection understand communicate listen learn 
• \ ' knowledge accountability connection self-reflection 
f-reflection curiosity challenge create achievement 
iiderstand communicate listen learn access quality 
dlect curiosity challenge create achievement 
ity challenge create achievement connection 
communicate listen learn action understand 
knowledge accountability connection access 
If) genu ity self-reflection educate action 

r ' G 4 ywledge accountability connection self- 

uh wi&L ^ intellect curiosity challenge connection 

/^ess quality innovation success ingenuity 
j derstand communicate listen learn 

:en learn access quality innovation 
e educate innovation success self- 
ion understand communicate listen 
. osity challenge create achievement 


Dnnection self-reflection understand educate action understand communicate listen learn action understand communicate listen learn access 


uality innovation success ingenuity curiosity challenge create achievement connection self-reflection understand communicate listen learn 


Policy Statement 

www.learningoutcomesassessment.org 



Table of Contents 


Higher Education Quality: Why Documenting Learning Matters.. ..3 

Introduction... 3 

Why Documenting Student Learning Matters 3 

What We Know About Effective Assessment Work 4 

Final Thoughts 6 

Resources 8 

NILOA National Advisory Panel 10 

About NILOA 1 1 


NILOA Mission 


The National Institute for Learning 
Outcomes Assessment’s (NILOA) primary 
objective is to discover and disseminate 
the ways that academic programs and 
institutions can productively use assessment 
data internally to inform and strengthen 
undergraduate education, and externally to 
communicate with policy makers, families, 
and other stakeholders. 
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Introduction 

The importance of assessing student learning in college has yet to capture the 
attention of policy makers or the public. Indeed, few outside the academy 
know what the phrase, student learning outcomes assessment, means. And 
yet the information outcomes assessment produces — when done well — 
is foundational to addressing some of the greatest challenges the country 
currently faces. 

Thirty years ago the assessment bandwagon began rolling across the landscape 
of American higher education. The movement was prompted in large part 
by the highly publicized 1983 federal report, A Nation at Risk, which argued 
American education needed to improve. This document was followed by a 
spate of others focused more squarely on higher education and the quality of 
the undergraduate experience. For three decades, institutions, accreditors, 
blue ribbon commissions, faculty, staff and others have invested considerable 
time and energy advancing efforts to document and enhance what students 
know and can do as a result of their studies. 

What do we have to show for all this activity? Well, more than many realize, 
but not nearly enough. 

Why Documenting Student Learning Matters 

Virtually everyone agrees that what students learn in college is central 
to subsequent success and satisfaction in life, to the nations economic 
competitiveness and productivity, and to building healthy and civically 
engaged communities. For this and many other reasons, the stakes have never 
been higher in terms of making sure college graduates acquire the knowledge 
and proficiencies needed to be self-sufficient and civically responsible. 

More recently a whole host of concerns has pushed questions about 
collegiate quality higher on the national agenda. Large numbers of students 
grapple with troubling levels of debt. Has their investment been well placed? 
Institutions themselves — both public and private — are stressed by financial 
problems, forcing cuts in the programs and services students need, especially 
those historically underserved by colleges and universities. What is the impact 
on learning? In addition, the spike in the number of new providers of higher 
learning along with accelerating technological advances make it possible for 
students to acquire postsecondary credentials without ever meeting their 
instructors in person or setting foot on a campus— a prospect that can raise 
questions about academic quality and integrity. 

These circumstances make educational quality a national priority. With so 
much in play, we should expect a groundswell of interest by faculty and staff 
as well as policy makers in assessing what students gain from their studies 
and using that information to enhance student attainment. But this is not 
the case. 



Virtually everyone agrees 
that what students learn in 
college is central to subsequent 
success and satisfaction in 
life, to the nations economic 
competitiveness and 
productivity, and to building 
healthy and civically engaged 
communities. 


“Employers recently have 
been asking for more than 
just a transcript in order to 
determine if students have 
learned what they need to 
know and do to function 
effectively.” 

-George P. Pernsteiner, 
President, State Higher 
Education Executive Officers 
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True enough, there has been some progress. Multiple studies indicate that 
the vast majority of colleges and universities publish statements about the 
intended learning outcomes of their degree programs. Where they take the 
next step — generating and using evidence of those outcomes — the results 
are often salutary. But some institutions — among them those perceived 
as the most prestigious — have yet to publicly specify their expectations 
for student learning or report assessment results. And many rank and file 
faculty members resist the specification and assessment of learning goals as 
reductionist and demeaning. 

Some of this resistance may be justified, including the worry that the very 
process of explicitly defining and systematically examining student learning 
can unintentionally reduce a complex and creative process to a list of 
mindless, elementary tasks. Effective teaching and learning is not about 
getting students to master facts; it is about engaging them in ways that foster 
a genuine love of inquiry and a facility with analytical reasoning and other 
higher order proficiencies. The challenge is to represent and gather evidence 
of those proficiencies in concrete ways that preserve their complexity. 
Another common criticism is that relying on assessment tools and processes 
developed by external vendors implies that faculty members are not trusted 
or competent to do this on their own. 

To these and other reservations and hazards we say, Amen l 

Documenting learning and using that evidence to improve student and 
institutional performance is a challenging, complicated process. But 
acknowledging the difficulty of the work in no way diminishes the urgency 
of doing it. Indeed, student learning outcomes assessment — done well — is 
not just a powerful, potentially effective means to improve student success; 
it is an essential strategy for the higher education enterprise to respond 
successfully to the many challenges it faces. 

What We Know About Effective Assessment Work 



Documenting learning and 
using that evidence to improve 
student and institutional 
performance is a challengingy 
complicated process. 


“The assessment movement is 
poised to drive a sea change 
in what actually counts 
as meaningful, actionable 
evidence by putting students 5 
authentic work at the center 
of the assessment inquiry . 55 

^ Carol Geary Schneider, 
President, Association of 
American Colleges and 
Universities 


Since 2008 the National Institute for Learning Outcomes Assessment 
(NILOA) has been tracking what colleges and universities are doing to 
document and improve student performance and institutional effectiveness. 
Over this period three trends are evident: 

• Institutions are clearer about what they expect their students to know 

and be able to do and they are more willing to make these expecta- 
tions public; 

• A wider range of assessment tools and approaches is available; and 

• Most institutions are using multiple approaches to evaluate student 

accomplishment. 

In addition, many accrediting groups — especially pro gram- specific entities — 
are encouraging more nuanced, thoughtful ways to assess student learning. 

From our work in the field, NILOA has distilled five principles that if 
enacted in mission-relevant ways can spread and accelerate assessment work 
worthy of the promises colleges and universities make to their students, 
policy-makers, and the public. 
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1 . Develop specific, actionable learning outcomes statements. 

Learning outcome statements are most useful when they are crafted to inform 
effective educational policies and practices, not to meet compliance demands 
by external groups. When they are concrete and clear about the proficiencies 
students are to achieve, such statements provide reference points for student 
performance, not just for individual courses but the cumulative effects of a 
program of study. Doing this demands active, operational verbs to guide 
the design of assignments that motivate students to demonstrate the desired 
outcomes in a way that can be verified. Clear, specific statements describing 
desired outcomes also make it possible for faculty to align curriculum and 
pedagogy with intended proficiencies, which is essential to ensuring that 
a program is, indeed, achieving its purposes. Such statements also make it 
easier for students to understand and appreciate institutional and program 
expectations for their performance and how their learning will equip them 
to handle what they encounter after college. 

2. Connect learning goals with actual student assignments and work. 

Provosts tell us that the most meaningful, actionable evidence of student 
learning comes from course-based assessments embedded in regular 
assignments. Indeed, our NILOA surveys show that faculty-designed 
assignments are the primary vehicle through which students demonstrate 
that they know and can do what the institution or program specifies; they are 
used far more frequently than standardized tests. To underscore the critical 
role of assignments, NILOA has conducted a series of “charrettes” in which 
faculty from different fields discuss how to improve their assignments in ways 
that more accurately align with one or more intended proficiencies. The 
products of that process are now available in an online assignment library 
[www.assignmentlibrary.org] that illustrates how degree-level proficiencies 
such as those identified in the Degree Qualifications Profile can be both 
fostered and assessed through papers, projects, demonstrations, reports and 
other tasks that faculty require of students. The Association of American 
Colleges and Universities, drawing on its Essential Learning Outcomes, is 
sponsoring parallel work on “signature assignments” and its partnership 
with the State Higher Education Executive Officers coordinating the Multi- 
State Collaborative (MSC). Faculty participating in such efforts design 
assignments for individual courses and also work to sequence assignments 
and incorporate high-impact practices across a program of study and across 
transfer pathways. An essential feature of these approaches is a systematic 
examination of artifacts of authentic student learning. 

3. Collaborate with the relevant stakeholders, beginning with the faculty. 

Faculty engagement and ownership are essential if assessment and 
improvement efforts are to be effective. And here, too, significant challenges 
exist. Relatively few faculty members have experience designing clear, explicit 
course and program outcomes or assignments that directly elicit those 
outcomes. Not surprisingly, campuses that have made the most progress 
have invested in serious, sustained professional development and have hosted 
venues where faculty can come together to formulate and explore questions 
about their students’ learning. When undertaken collaboratively with others 
who work with students — such as student affairs staff and advisors — this 



Learning outcome statements 
are most useful when 
they are crafted to inform 
effective educational policies 
and practicesy not to meet 
compliance demands by 
external groups . 


“NILOA’s five principles for 
effective learning assessment 
should be central to every 
institution’s efforts to better 
understand what and how 
students are learning and 
using that information to 
further strengthen student 
and institutional outcomes.” 

^ Brian K. Bridges, Vice 
President, Research and 
Member Engagement, 
United Negro College Fund 
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kind of inquiry can move outcomes assessment from an “add on” to a process 
that is part and parcel to effective teaching and learning. That shift is critical 
to fostering a sense of collective responsibility for learning among all those 
who have a role in the educational process and to establishing systematic 
assessment as a shared professional norm. 

4. Design assessment approaches that generate actionable evidence 
about student learning that key stakeholders can understand and use to 
improve student and institutional performance. 

Colleges and universities have more information about students and their 
learning than ever before. But too few institutions use productively what 
they have in hand. In large part, this is because the evidence available is not 
translated into actions to enhance student accomplishment. The good news 
is that we have learned how some institutions do this well. 

• They ask questions about student performance to which faculty and 

others want answers. This means involving the right stakeholders — 
faculty, staff, students, governing board members, and others as 
appropriate — at the beginning of any assessment project to deter- 
mine the questions it needs to answer. 

• They build interest and momentum by creating occasions for people 
to work together to raise issues and questions they care and need 
to know more about in order to improve student engagement and 
learning. And they bring these same people back together to make 
sense of the findings and tease out their implications for action. 

• They present assessment results in transparent, understandable forms 
to the people who have a need to know and act on them. 

5. Focus on improvement and compliance will take care of itself. 

Assessment that comports with the four previous principles — employing 
integrated, stakeholder-responsive, action-oriented approaches — has the 
added, salutary effect of obviating the compliance mentality that often 
blunts the prospects for effective assessment efforts. Many schools trapped 
in this “culture of compliance” have either out-sourced the assessment 
process or hired professional staff to respond to what are often ambiguous 
and conflicting demands from accreditors and other external actors. And too 
often the results of compliance-driven assessment — which frequently take the 
form of standardized tests — are unconnected to policies and practices that 
matter to desired outcomes. In contrast, assessment motivated by genuine 
institutional needs and faculty priorities for improving teaching typically 
yields evidence that is valued and more likely to be used. When college and 
universities do this right, assessment becomes embedded in the regular daily 
work of the academy and external actors like regional accreditors are more 
than satisfied. 

Final Thoughts 


Clarifying and documenting what students know and can do and using this 
information to improve student and institutional performance are essential 
because students need a postsecondary education that will prepare them to 
meet the challenges of the 21 st century. In addition, educators have work to 



Assessment that comports with 
the four previous principles — 
employing integrated \ 
stakeholder-responsive, action- 
oriented approaches — has 
the added, salutary effect 
of obviating the compliance 
mentality that often blunts 
the prospects for effective 
assessment efforts. 


“NILOAs five principles are 
clear, practical, and useful. As 
to the fifth principle, ‘Focus on 
improvement and compliance 
will take care of itself, 5 this 
accreditor says Amen 5 ! 55 

-Mary Ellen Petrisko, 
President, WASC Senior 
College and University 
Commission 
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do in building and maintaining public trust and demonstrating institutional 
integrity. It is no longer beyond the capacity of a college or university to 
articulate expectations for learning, to document student progress toward 
these expectations, and to use the resulting evidence to improve student 
success. Doing this job and doing it well is within our grasp. Failing to do 
so shortchanges our students and the many others who have a major stake 
in the quality of higher education. Equally important, we in the academy 
owe it to ourselves. 
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